Notes on Linear Minimum Mean Square Error Estimators

Some connections between linear minimum mean square error estimators, maximum output SNR filters and the least square solutions are presented. The notes have been prepared to be distributed with EE 503 (METU, Electrical Engin.) lecture notes. 1 Linear Minimum Mean Square Error Estimators The following signal model is assumed: r = Hs + v (1) Here r is a N × 1 column vector denoting the observations. In this model, s is the desired signal vector, v is the interference vector on the observations. We assume that s and v are uncorrelated as in all stochastic filtering applications. It can be noted that the observations are linear combinations of desired quantities and interference. Here the matrix H is an N × M matrix (N can be less than or greater than M , hence the system can be under or over determined). H can be considered as the channel or the observation system or a linear combiner for the modes where the modes are the columns of H. The column vector s contains M entries each of which is to be estimated. In class, we have derived the optimal linear minimum mean square error estimator as the solution of the following system of equations: RrW = Rrs (2) Here Rr = E{rrH} is the auto-correlation matrix of the observations and Rrs = E{rsH} is the cross-correlation matrix of observations and the desired variables. The minimum mean square error estimate for s is given as follows: ŝ = Wr (3) The error covariance matrix for the minimum error estimator is as shown below: Re = E{(s− ŝ) } {{ } e (s− ŝ)) } {{ } eH } = E{e(sH − rHW)} (a) = E{esH} − E{erH} } {{ } 0 W = E{(s−WHr)sH} = Rs −WRrs (4) The zero matrix on the right hand side of equation shown with (a) is due to the orthogonality condition of the optimal estimator. Next, we explicitly calculate the estimator in terms of H, Rs and Rv matrices. The following can be easily verified: Rr = HRsH + Rv Rrs = HRs (5)